1. Reservation of Copyright
The disclosure of this patent document contains material which is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the U.S. Patent and Trademark Office patent files or records, but otherwise reserves all copyright rights whatsoever.
2. Field of the Invention
The present invention relates to a machine vision system for identifying and assessing (i.e., characterizing) features of an article. The assessments may include determining the position of one or more features and/or inspecting features of the object to determine if they conform to a set of constraints. An aspect of the present invention relates to a system for developing feature assessment programs. The resulting programs, when deployed, can be used to inspect parts and/or provide position information for guiding automated manipulations of such parts.
3. Description of Background Information
Machine vision systems play an important role in automated systems. Cameras are used to obtain images of articles, and image processing is performed to identify features of the article. Further image processing may be performed to identify the article's position, measure its dimensions, and/or to check for article defects. Image processing results may then be used to aid in the control of automated systems, such as factory equipment, including, for example, an industrial controller, a robotic arm, or a positioning table. Such machine vision systems may aid in the inspection, assembly, and/or handling of various types of articles, parts, and devices, including automotive parts (e.g., fuses, gaskets, and spark plugs), electrical components (e.g., connector pins, keyboards, LED, LCD, VFD displays), medical and pharmaceutical products (e.g., disposable test kits, syringes, needles, and date-lot codes), and consumer products (e.g., razor blades and floppy disks).
Whenever a new article, part, or device is inspected by a given machine vision system, the vision processing component of the machine vision system will usually be modified and provided with a new inspection program for the new item. The vision processor will typically include a different visual inspection program for each type of device. For example, a system for automatically assembling (mounting) surface-mounted devices (SMDs) may have a vision processor which uses a unique visual inspection program for assessing each SMD as part of the mounting process.
Many SMD placement manufacturers will use a vision processor in their automated SMD placement system, in order to increase production rates and to accommodate accuracy demands associated with SMDs having dense, fine-pitch leads. In such systems, accurate siting of SMDs is accomplished using a different inspection program for each type of SMD involved. For example, COGNEX's SMD Placement Guidance Package provides unique respective software programs for large-leaded device inspections, front lit small-leaded device inspections, back lit small-leaded device inspections, front lit chip inspections, and so on. Further information about such software systems and subsystems is provided in the COGNEX 4000/5000 SMD Placement Guidance Package User's Manual Release 3.1, PN590-1039, Cognex Corporation (1996), the content of which is hereby expressly incorporated herein by reference in its entirety.
New or odd form SMDs which do not fall into any one of the categories for which inspection programs are already available require that a new inspection program be developed. The difficulty, however, with creating a new inspection program is that sophisticated programming skills are required, and there are usually delays in the development and perfection of a new inspection program for a new or odd-form SMD. New inspection programs will not be immediately available whenever a new or odd-form SMD emerges. For this reason, manufacturers or assemblers may consider delaying the introduction of a new or odd form SMD into a product, force-fitting an existing inspection program to accommodate the new device, or otherwise mounting (e.g., manually mounting) the SMD.
The complications associated with creating a new vision inspection program increase for multiple field of view (MFOV) inspections, i.e., inspections performed using more than one field of view (FOV). The geometry of some devices will make it impossible to successfully run an inspection program using only one FOV, i.e., a single field of view (SFOV). MFOVs are required either when the device is very large or when the device has fine details that must be picked up in the image data. When the device has fine details, the resolution of the image of the device must remain high, which requires that the physical space corresponding to each field of view be much smaller than the total device.
In order to accommodate such limitations in the field of view size, the scene is divided into several fields of view. A scene is an area of physical space, which may include the device and a portion of the background, that should be considered in order to perform a successful inspection. The scene may comprise the complete device and background information surrounding the device, or it may comprise only portions of the device that are needed to perform the inspection. When the complete scene cannot be captured with one image within the field of view of a single camera, while maintaining adequate resolution of the image, the scene must be divided into several such images or fields of view called multiple fields of view (MFOVs).
The lead pitch and other small features of today's and upcoming SMDs are decreasing in size. Such fine SMD features will require higher image resolution, which will likely cause MFOV inspections to become more commonplace.
One type of conventional article assessment development system includes standard development software provided by Cognex, which includes a library of high-level vision software and image processing tools. It also includes system software designed to facilitate code development and debugging.
In order to create a vision-based inspection program with these COGNEX products, a user writes a C-language program that connects the software blocks appropriate for a desired feature identification and assessment.
Another type of development system which may be used to create a vision-based system for identifying features of an article and assessing those features is the CHECKPOINT system, provided by COGNEX. The CHECKPOINT system uses a graphical user interface for aiding in the development of vision programs. A developer may utilize the CHECKPOINT interface to combine high-level vision, I/O and operator interface tools with conventional programming elements. Such conventional programming elements can include menu-selected program statements, such as conditional statements (If/Then, If/Else), assigning a value or result of an expression to a variable name (Set/Set Reference), defining conditional or iterative looping (For/While/Break/ Next), using or invoking a function, or ending a function and returning a value (Call/Return), calculating the value of an expression (Calc), jumping to a statement (Go to/Label), and providing program comments (Notes).
Such program statements are built automatically from a dialog input box. As these system commands are entered, the CHECKPOINT system checks the entered commands for syntax errors. The CHECKPOINT system is also provided with dialog boxes for inputting important tool parameters, windows for accessing data and functions of the checkpoint development system, and debugging capabilities. The system is also provided with a mechanism for simple editing of a vision routine by using statements to cut, copy, and paste utilities.
The COGNEX standard development software is flexible and facilitates the development of inspection programs. On the other hand, the CHECKPOINT system uses a more simplified user interface and provides more guidance to aid in faster and easier development of inspection programs. There is a need for an improved inspection program development system which will be flexible and sophisticated, and will facilitate quicker development of inspection programs without requiring a high level of programming proficiency.
4. Definitions of Terms
The following term definitions are provided to assist in conveying an understanding of the various exemplary embodiments and features disclosed herein.
Class:
A data structure descriptive of (defining basic attributes of) objects which may be instantiated therefrom.
Computer-Readable Medium:
Physical material, including, for example, a disk, tape, a RAM, a ROM, and so on for storing computer-readable information. A computer-readable medium may comprise one or more data storage media, and, if plural data storage media are utilized, those media may comprise different types of media.
Device Space:
A coordinate system using physical units (e.g., microns) to describe features of a device with respect to a reference point fixed with respect to the device.
Field of View (FOV) or Single Field of View (SFOV):
Image data representing an image obtained by a single image acquisition. Usually, a single field of view comprises image data corresponding to the area of physical space that a particular camera acquires as image data.
Image Space:
A coordinate system using virtual units (e.g., pixels) to describe features of a device.
Install:
To set up and prepare for operation. For example, when a step object is installed, it is set up and prepared for operation.
Instance:
An object created (by allocating memory therefor) from a particular class.
Instantiate:
To create an object which is an instance of a class.
Library:
A collection of routines, or a collection of objects, modules, or other entities which each comprise one or more routines.
Module:
A collection of routines and data structures. A module will include an interface, which comprises constants, data types, variables, and routines accessible by other modules or routines, and code (accessible only to the module) which facilitates implementation of the routines into the module.
Multiple field of view (MFOV) inspection:
An inspection process performed using more than one FOV. Partial inspection processes may be performed on each field of view, and all partial inspections then combined to create a final result.
Object:
A variable comprising both routine(s) and data treated discretely from the routine(s).
Physical Space:
A coordinate system using physical units to describe features of a device with respect to a reference point that is independent of the device.
Routine:
A section of code which can be invoked (executed) within a program being run by a computer.
Scene:
Image data corresponding to an area of physical space used to perform an inspection. A scene may include all of an article and a portion of the background.